Image capturing apparatus for performing authentication of a photographer and organizing image data for each photographer and control method thereof

ABSTRACT

An image capturing apparatus, comprises an image capturing unit configured to capture an image; a detection unit configured to detect an object in a captured image; a first authentication unit configured to perform authentication on a photographer using first dictionary data in which feature information of the photographer is registered; a second authentication unit configured to perform authentication on the object in the captured image using second dictionary data in which feature information of the object is registered for each photographer; and a switching unit configured to switch the second dictionary data to be used by the second authentication unit in accordance with the photographer authenticated by the first authentication unit.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an image capturing apparatus having afunction of performing authentication on an object and a photographer.

Description of the Related Art

Some recent image capturing apparatuses such as digital cameras havefunctions of performing authentication on specific objects, including aface recognition function and a face dictionary function. The facerecognition function, which is for identifying a face area of a personfrom a captured image, is applied to functions including a function ofdetermining whether a person is included in an object and a function ofautomatically focusing on a person, and is used for enabling aphotographer to comfortably perform shooting. The face dictionaryfunction is for identifying a person by extracting a feature amount of aperson's face recognized with the face recognition function andcomparing the extracted feature amount with a feature amount of apre-registered person, and is applied to organization of image dataafter shooting and a search for the image data.

As a technique applying these functions, for example, Japanese PatentLaid-Open No. 2009-290260 recites a technique using a sub-camera forshooting an object located on the photographer side, by which thephotographer in an image that is shot by the sub-camera is recognizedand a camera setting is automatically switched in accordance with achange of the photographer.

For example, if a camera equipped with the face dictionary function isshared by a plurality of users, even a photographer who is a strangerwill be authenticated as a specific object if this photographer isregistered in the face dictionary. Moreover, if there is a photographerhaving a similar feature amount of the face, it is possible that thisphotographer will be misidentified as a completely unknown person.

Furthermore, although an appropriate shooting environment can beprovided for each photographer with the technique recited in JapanesePatent Laid-Open No. 2009-290260, this technique is not useful fororganization or selection of image data after shooting.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of theaforementioned problems.

In order to solve the aforementioned problems, the present inventionprovides an image capturing apparatus, comprising: an image capturingunit configured to capture an image; a detection unit configured todetect an object in a captured image; a first authentication unitconfigured to perform authentication on a photographer using firstdictionary data in which feature information of the photographer isregistered; a second authentication unit configured to performauthentication on the object in the captured image using seconddictionary data in which feature information of the object is registeredfor each photographer; and a switching unit configured to switch thesecond dictionary data to be used by the second authentication unit inaccordance with the photographer authenticated by the firstauthentication unit.

In order to solve the aforementioned problems, the present inventionprovides a control method of an image capturing apparatus, comprising: adetection step of detecting an object in a captured image; a firstauthentication step of performing authentication on a photographer usingfirst dictionary data in which feature information of the photographeris registered; a second authentication step of performing authenticationon the object in the captured image using second dictionary data inwhich feature information of the object is registered for eachphotographer; and a switching step of switching the second dictionarydata to be used in the second authentication step in accordance with thephotographer authenticated in the first authentication step.

According to the present invention, execution of unnecessary objectauthentication for a photographer who is a stranger can be suppressed,and misidentification of a photographer having a similar feature amountof the face as a completely unknown person can also be suppressed.

Furthermore, image data can be organized and selected for eachphotographer using a person's name.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration of an image capturingapparatus in the present embodiment.

FIG. 2 is a functional block diagram for realizing a face recognitionfunction and a face dictionary function in the present embodiment.

FIG. 3 is a diagram showing a configuration of a face dictionarydatabase in the present embodiment.

FIG. 4 is a diagram showing a data format of face dictionaries in a facedictionary database in the present embodiment.

FIG. 5 is a flowchart showing facial authentication processing performedby the image capturing apparatus with respect to a photographer and anobject in the present embodiment.

FIG. 6 is a diagram illustrating a shooting operation using a maincamera and a sub-camera of the image capturing apparatus in the presentembodiment.

FIG. 7 is a diagram illustrating facial authentication processingperformed by the image capturing apparatus with respect to aphotographer and an object in the present embodiment.

FIG. 8 is a flowchart showing facial authentication processing performedby the image capturing apparatus with respect to a photographer in thepresent embodiment.

FIG. 9 is an outline drawing showing a configuration of an imagecapturing apparatus in a second embodiment.

FIG. 10 is a functional block diagram for realizing authenticationprocessing performed by the image capturing apparatus with respect to aphotographer in the second embodiment.

FIG. 11 is a flowchart showing authentication processing performed bythe image capturing apparatus with respect to a photographer in thesecond embodiment.

FIG. 12 is a diagram showing an exemplary UI screen for allowing a userto designate a specific photographer in an image capturing apparatus ina third embodiment.

FIG. 13 is a diagram illustrating facial authentication processingperformed by an image capturing apparatus with respect to a photographerin a fourth embodiment.

FIG. 14 is a flowchart showing facial authentication processingperformed by the image capturing apparatus with respect to aphotographer in the fourth embodiment.

FIG. 15 is a flowchart showing facial authentication processingperformed by the image capturing apparatus with respect to aphotographer in the fourth embodiment.

FIG. 16 is a diagram showing an exemplary UI screen for allowing aphotographer to register his/her own information in a face dictionary inan image capturing apparatus in a fifth embodiment.

FIG. 17 is a flowchart showing processing in which a photographerregisters his/her own information in a face dictionary in the imagecapturing apparatus in the fifth embodiment.

FIGS. 18A and 18B are diagrams showing exemplary UI screens for allowinga user to copy an existing face dictionary to another device with animage capturing apparatus in a sixth embodiment.

FIGS. 19A and 19B are diagrams showing exemplary UI screens of a copydestination image capturing apparatus in the case where a user copies anexisting face dictionary to another device with the image capturingapparatus in the sixth embodiment.

FIG. 20 is a diagram showing a data format of a face dictionary for eachphotographer in the image capturing apparatus in the sixth embodiment.

FIG. 21 is a flowchart showing processing for copying a face dictionaryindividually for each photographer in the image capturing apparatus inthe sixth embodiment.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will be described in detail below.The following embodiments are merely examples for practicing the presentinvention. The embodiments should be properly modified or changeddepending on various conditions and the structure of an apparatus towhich the present invention is applied. The present invention should notbe limited to the following embodiments. Also, parts of the embodimentsto be described later may be properly combined.

First Embodiment

Hereinafter, an embodiment will be described in which the presentinvention is applied to a digital camera or the like having a facerecognition function and a face dictionary function, for example, butthe present invention is not limited thereto. The present invention canalso be applied to information processing apparatuses such as mobilephones, portable media players, so-called tablet devices, and personalcomputers, for example.

Apparatus Configuration

An outline of a configuration and functions of an image capturingapparatus in the present embodiment will now be described with referenceto FIG. 1.

In an image capturing apparatus 101 shown in FIG. 1, a CPU 102 is acontrol unit for controlling the overall apparatus and performingvarious kinds of information processing. A timer 103 is a timer forrecording shooting date and time of an image. A shutter button 104 is anoperation unit for giving a shooting instruction. A display unit 105 isan LCD screen or the like for displaying an object at the time ofshooting and reproducing an image after shooting, and is capable ofaccepting a user input operation via a touch panel. A main camera 106 isa camera module which performs shooting of the object side. A sub-camera107 is a camera module which performs shooting of an object on thephotographer side. A memory 108 temporarily holds data of images shot bythe aforementioned camera modules. A memory interface 109 can access arecording medium, such as a memory card, that is attached to theapparatus body, and stores shot image data and the like. A biometricsunit 110 is, for example, a fingerprint sensor, a microphone for voicerecognition processing, a finder camera for iris pattern determination,or the like, and is used for reading biological information used forperforming authentication on a photographer.

Object Authentication Module

Next, a configuration of a software module by which the image capturingapparatus in the present embodiment realizes a face recognition functionand a face dictionary function will be described with reference to FIG.2.

In FIG. 2, a signal processing unit 201 converts optical images receivedby the main camera 106 and the sub-camera 107 into electric signals. Theelectric signals obtained as a result of photoelectric conversion by thesignal processing unit 201 are converted into digital information thatcan be handled as image data by an image processing unit 202.

The image data generated by the image processing unit 202 is convertedinto an image file in a format to be stored in a computer or the like byan image file generation processing unit 203, and is recorded in thememory 108.

A face detection processing unit 204 searches for a portion that can berecognized as a person's face in the image data generated by the imageprocessing unit 202. A facial authentication processing unit 205,references a face dictionary database (DB) 206 in which featureinformation (hereinafter referred to as “feature amount”) of a person'sface is registered, thereby performs authentication of the name of theperson corresponding to the face recognized by the face detectionprocessing unit 204, and presents an authentication result.

A face dictionary DB 206 is a set of data loaded into the memory 108 andthe feature amount of a person's face recognized by the face detectionprocessing unit 204 is registered therein in association with a person'sname.

The above is the configuration of the software module having a generalobject authentication function, and in the present embodiment, aphotographer authentication processing unit 207 and a face dictionaryswitching processing unit 208 are additionally provided.

The photographer authentication processing unit 207, analyzes thefeature amount of the face of a photographer based on the image shot bythe sub-camera 107, and identifies the photographer by referencing theface dictionary DB 206, for example. Here, the methods for identifyingthe photographer also include, for example, determination usinginformation that is directly input by the photographer via a touch panelor the like, or biological information that is input in the biometricsunit 110, in addition to the aforementioned method.

The face dictionary switching processing unit 208 switches the facedictionary DB 206 that is to be referenced by the facial authenticationprocessing unit 205, in accordance with the photographer identified bythe photographer authentication processing unit 207.

Configuration of Face Dictionary DB

Next, a configuration of a face dictionary for each photographer managedin the face dictionary DB 206 in the present embodiment will bedescribed with reference to FIG. 3.

In FIG. 3, in the face dictionary DB in the image capturing apparatus101, a face dictionary 302 for a photographer X, a face dictionary 303for a photographer Y, and a face dictionary 304 for a photographer Z areprepared in association with the respective photographers.

In the face dictionaries 302, 303, and 304 for the respectivephotographers, feature amounts of different objects are registered. Thatis to say, the face dictionary 302 for the photographer X holds thefeature amounts of an object A (305), an object B (306), and an object C(307). The face dictionary 303 for the photographer Y holds the featureamounts of the object A (308) and the object C (309). The facedictionary 304 for the photographer Z holds the feature amounts of theobject B (310) and the object D (311).

The face dictionaries 302 to 304 for the respective photographers areswitched by the face dictionary switching processing unit 208 andreferenced by the facial authentication processing unit 205, inaccordance with the photographer authenticated by the photographerauthentication processing unit 207.

Data Format of Face Dictionary

Next, a data format of the face dictionaries held in the face dictionaryDB 206 in the present embodiment will be described with reference toFIG. 4.

In FIG. 4, reference numeral 401 denotes the face dictionary DB, andreference numeral 402 denotes ID numbers for management on a programthat are not to be shown to photographers, and each ID number indicatesa unit of a person (object) based on the association between the featureamount of the face of the object, which will be described later, and thename registered in the face dictionary. Reference numeral 403 denotesinformation extracted as the feature amount of the face, and is used toidentify a person by comparing this information with the feature amountsof people who are objects. Reference numerals 404, 405, and 406 areregistered name tables constituted by names (registered names) that areuniquely assigned to the respective people managed in the facedictionary DB 206, and a table is prepared for each photographer. Thephotographer can input a character string of a name by operating thetouch panel on the display unit 105.

Upon the face dictionary (photographer) being switched by the facedictionary switching processing unit 208, the registered name table fora target photographer is loaded into the memory 108 again and referencedby the facial authentication processing unit 205.

Facial Authentication Processing

Next, facial authentication processing performed by the image capturingapparatus with respect to a photographer and an object in the presentembodiment will be described with reference to FIG. 5. Note that thisprocessing is realized as a result of a program recorded in anonvolatile memory such as a ROM being deployed to the memory 108 andexecuted by the CPU 102.

In FIG. 5, in step S501, the CPU 102 performs authentication processingwith respect to the photographer with the photographer authenticationprocessing unit 207, advances the processing to step S502 if thephotographer is authenticated, and advances the processing to step S503if not.

In step S502, the face dictionary for the photographer authenticated bythe face dictionary switching processing unit 208 is loaded from theface dictionary DB 206 into the memory 108, and the CPU 102 canreference the face dictionary at the facial authentication processingunit 205.

On the other hand, in step S503, the CPU 102 causes a common facedictionary that does not identify a photographer to be loaded into thememory 108 or causes a face dictionary not to be loaded (i.e., not to beused), and it is determined whether the common face dictionary is loadedor a face dictionary is not loaded, in accordance with the initialsettings of the program.

In step S504, the CPU 102 determines whether or not the shutter button104 has been pressed and shooting processing has been performed,advances the processing to step S505 if the shutter button 104 has beenpressed, and returns to step S501 if not.

In step S505, the CPU 102 determines whether a face dictionary has beenloaded into the memory 108 and can be referenced, advances theprocessing to step S506 if the face dictionary can be referenced, andends the processing if not.

In step S506, the CPU 102 analyzes captured image data (through-the-lensimage data) with the face detection processing unit 204, advances theprocessing to step S507 if it is recognized that a face exists in theimage, and ends the processing if not.

In step S507, the CPU 102 references the face dictionary with the facialauthentication processing unit 205, performs authentication of the nameof the person having the face recognized at the face detectionprocessing unit 204, and records an authentication result as imageattribute information. Here, the image attribute information includes aphotographer's name and an object's name registered in the facedictionary for the photographer.

Note that a configuration may also be employed in which a frame isdisplayed so as to surround the face of an object and the name of theobject is displayed in the vicinity of the frame when an image isreproduced, based on the attribute information recorded in step S507.Furthermore, before the shutter button is pressed, the aforementionedfacial authentication processing may be performed on a through-the-lensimage such that the frame and the name are displayed as mentioned above.

As described above, control is performed such that the face dictionarycorresponding to the photographer at the time of shooting is referenced.

Facial Authentication Processing

Next, facial authentication processing performed by the image capturingapparatus with respect to a photographer and an object in the presentembodiment will be described with reference to FIGS. 6 to 8.

In FIG. 6, reference numeral 602 denotes an image shot by the sub-camera107 (hereinafter referred to as a “sub-camera image”), and a frame 603is displayed around the face of the photographer detected by the facedetection processing unit 204. Reference numeral 604 denotes an imageshot by the main camera 106 (hereinafter referred to as a “main cameraimage”), and frames 605 to 607 are displayed around the faces of aplurality of objects detected by the face detection processing unit 204.

Upon the photographer's face being detected in the sub-camera image 602by the face detection processing unit 204, the photographerauthentication processing unit 207 performs authentication processingwith respect to the photographer, the face dictionary switchingprocessing unit 208 switches the face dictionary to the face dictionaryfor the authenticated photographer, which is loaded into the memory 108.Then, the facial authentication processing unit 205 references the facedictionary loaded into the memory 108, and performs facialauthentication processing with respect to the objects existing in themain camera image 604.

Note that, as described above, a configuration may also be employed inwhich the names of the objects are displayed in the vicinity of theframes 605 to 607 of the objects' faces in the main camera image 604,based on the result of the aforementioned facial authenticationprocessing with respect to the objects. Furthermore, the aforementionedfacial authentication processing may be performed on a through-the-lensimage before the shutter button is pressed such that the frames and thenames are displayed as mentioned above. Moreover, a configuration mayalso be employed in which the sub-camera image 602 is displayed in aportion of the main camera image 604, and the photographer's name isdisplayed in the vicinity of the frame 603 of the photographer's name inthe sub-camera image 602 based on the result of the aforementionedfacial authentication processing with respect to the photographer.

FIG. 7 shows an exemplary configuration of the face dictionaries used inthe facial authentication processing in the present embodiment.

In FIG. 7, as in FIG. 3, the prepared face dictionaries for therespective photographers that serve as second dictionary data include aface dictionary 706 for a photographer X, a face dictionary 707 for aphotographer Y, and a face dictionary 708 for a photographer Z.Furthermore, in FIG. 7, a face dictionary 702 for photographerauthentication that serves as first dictionary data is prepared, andfeature amounts 703, 704, and 705 of the faces of the photographers X,Y, and Z who are registered in the face dictionaries for thephotographers are registered.

The face dictionaries 706 to 708 for the respective photographers aremanaged independently of the photographers and loaded into the memory108 so as to be able to be referenced in common by any photographer.

FIG. 8 is a flowchart showing facial authentication processing withrespect to the photographer in step S501 in FIG. 5.

In FIG. 8, in step S801, the CPU 102 shoots an image with the sub-camera107.

In step S802, the CPU 102 determines whether or not a face portionexists in the sub-camera image 602, using the face detection processingunit 204. If it is determined in step S802 that the face portion exists,the processing proceeds to step S803, and if it is determined that aface portion does not exist, the processing ends.

In step S803, the CPU 102 references the face dictionary 702 forphotographer authentication with the facial authentication processingunit 205.

In step S804, the CPU 102 causes the photographer authenticationprocessing unit 207 to determine whether or not the face recognized bythe face detection processing unit 204 is registered in the facedictionary for photographer authentication, and advances the processingto step S805 if the face is registered, and if not, the CPU 102 does notupdate the photographer and ends the processing ends.

In step S805, the CPU 102 records the person authenticated by thephotographer authentication processing unit 207 as a program variable inthe memory 108 such that the authenticated person can be used as aphotographer in later processing.

Second Embodiment

Next, authentication processing performed by an image capturingapparatus with respect to a photographer in a second embodiment will bedescribed.

In the present embodiment, authentication processing is performed withrespect to a photographer using a fingerprint sensor in the biometricsunit 110.

FIG. 9 is an outline drawing of a back face of the image capturingapparatus in the second embodiment.

In FIG. 9, a liquid crystal panel 902 serving as the display unit 105 isdisposed on a back face 901 of the apparatus body. The liquid crystalpanel 902 is used for, for example, display of an image shot by the maincamera 106, display of an operation screen for camera settings or thelike, and display of image data stored in a memory card or the like. Aring-shaped four-direction button 903, a determination button 904, acancel button 905, a fingerprint sensor 906, and a menu button 907 aredisposed in the vicinity of the liquid crystal panel 902. With thefour-direction button 903, the user can input various instructions tocontrol the operation screen displayed on the liquid crystal panel 902,to change the display screen, and the like, by pressing down thefour-direction button 903 in up-and-down and left-and-right directions.By pressing down the determination button 904, the user can inputselection of an item on the operation screen displayed on the liquidcrystal panel 902 and an instruction. By pressing down the cancel button905, the user can input an instruction to cancel an item on theoperation screen displayed on the liquid crystal panel 902. Thefingerprint sensor 906 can read a fingerprint as a result of a pad of aphotographer's finger sliding thereon. By pressing down the menu button907, the user can display a camera setting menu for image shooting andreproduction on the liquid crystal panel 902.

With the configuration in FIG. 9, when the photographer operates theapparatus, the photographer can be subjected to authentication bycausing the fingerprint sensor 906 to read the photographer'sfingerprint.

FIG. 10 shows a configuration of a software module of a photographerauthentication processing unit in the image capturing apparatus in thesecond embodiment, corresponding to the photographer authenticationprocessing unit 207 in FIG. 2.

In FIG. 10, a biometrics processing unit 1001 identifies a photographerby referencing a biological information DB 1002 regarding biologicalinformation including the fingerprint that is read by the fingerprintsensor 906. In the biological information DB 1002, biologicalinformation for all photographers registered in the apparatus isregistered. The biological information DB 1002 is deployed to the memory108 so as to be able to be referenced by the biometrics processing unit1001, and a person who matches the biological information that is inputfrom the biometrics unit 110 is recorded as a program variable in thememory 108 so as to be able to be used as a photographer in laterprocessing.

Although the present embodiment describes an example of using afingerprint as biological information, any other kind of information bywhich a person can be identified, such as a vein pattern, an irispattern, and a voice print, may also be used, and the biometrics unit110 and the biological information DB 1002 that are appropriate for thebiological information to be used will be prepared.

FIG. 11 is a flowchart showing authentication processing with respect tothe photographer in step S501 in FIG. 5.

In FIG. 11, in step S1101, the CPU 102 reads biological informationusing the fingerprint sensor 906 serving as the biometrics unit 110.

In step S1102, the CPU 102 analyzes the read biological informationusing the biometrics processing unit 1001.

In step S1103, the CPU 102 determines whether or not the analyzedbiological information is registered in the biological information DB1002, advances the processing to step S1104 if the analyzed biologicalinformation is registered, and if not, the CPU 102 does not update thephotographer and ends the processing.

In step S1104, the CPU 102 records a person who matches the biologicalinformation registered in the biological information DB 1002 as aprogram variable in the memory 108 so as to be able to be used as aphotographer in later processing.

Other configurations of the present embodiment are similar to those ofthe first embodiment, and the description thereof will be omitted.

Third Embodiment

Next, face dictionary selection processing performed by the imagecapturing apparatus with respect to a photographer in a third embodimentwill be described.

In the present embodiment, processing for selecting a face dictionaryfor a photographer is performed by the photographer inputting apredetermined operation.

FIG. 12 shows an exemplary operation screen 1201 displayed on a liquidcrystal panel 902 in FIG. 9, and the operation screen 1201 is displayedby a user pressing down a menu button 907 or a four-direction button 903in FIG. 9.

Selection buttons 1202, 1203, and 1204 allow the user to designate aphotographer X, a photographer Y, or a photographer Z, and the names ofthe people who have been registered in advance as photographers aredisplayed. By pressing down the four-direction button 903, the user canswitch the photographer to be selected.

Upon a return (cancel) button 1205 being selected by the user, a setstate configured on the operation screen 1201 is deemed to be invalidand this set state is not reflected.

Upon an OK (set) button 1206 being selected by the user, a set stateconfigured on the operation screen 1201 is deemed to be valid and thisstate is reflected in later processing.

When a certain person uses the image capturing apparatus, thephotographer can be identified from among candidates displayed on theoperation screen 1201 by initially operating the menu button 907 todisplay the operation screen 1201 and thereafter operating thefour-direction button 903 and the OK button 1206.

Other configurations of the present embodiment are similar to those ofthe first embodiment, and the description thereof will be omitted.

Fourth Embodiment

Next, facial authentication processing performed by an image capturingapparatus with respect to a photographer in a fourth embodiment will bedescribed.

In the present embodiment, in the case where a photographer cannot beidentified in the above-described embodiments, facial authenticationprocessing is performed with respect to the photographer using the faceof an object detected in an image shot by the main camera 106. Here,“the case where a photographer cannot be identified” refers to, forexample, the case where a self-timer is used to perform shooting or thecase of temporarily having another person press the shutter button, anda situation where the photographer himself/herself is included in themain camera image is envisioned.

FIG. 13 is a diagram for illustrating facial authentication processingwith respect to the photographer in the fourth embodiment.

In FIG. 13, in self-timer shooting, a photographer 1302 configuressettings of the self-timer on the image capturing apparatus 101,thereafter moves onto the main camera side, and appears in an image 1303together with other objects 1304 to 1306. In this case, since no oneexists on the sub-camera side at the moment of shutter release, thephotographer cannot be identified using an image shot by the sub-camera107 as in the first embodiment, for example. For this reason,authentication of the photographer is performed based on peoplerecognized in the main camera image.

FIG. 14 is a flowchart showing facial authentication processingperformed by the image capturing apparatus with respect to thephotographer in the fourth embodiment.

In FIG. 14, in step S1401, the CPU 102 shoots an image using thesub-camera 107 and determines whether or not a face area exists in thesub-camera image with the face detection processing unit 204. If it isdetermined in step S1401 that a face area exists, the CPU 102 advancesthe processing to step S1402, and if it is determined that a face areadoes not exist, the CPU 102 advances the processing to step S1405.

In step S1402, the CPU 102 causes the facial authentication processingunit 205 to reference the face dictionary DB 206 and thereby determineswhether or not the photographer has been identified. If the photographerhas been identified in step S1402, the CPU 102 advances the processingto step S1403, and if the photographer cannot be identified since theface recognized in step S1401 is not registered in the face dictionaryDB 206, the CPU 102 advances the processing to step S1405.

The processing in step S1403 and subsequent steps is the same as that instep S502 and subsequent steps in FIG. 5.

On the other hand, in step S1405, the CPU 102 shoots an image using themain camera 106, and determines whether or not a face area exists in themain camera image with the face detection processing unit 204. If it isdetermined in step S1405 that a face area exists, the CPU 102 advancesthe processing to step S1406, and if it is determined that a face areadoes not exist, the CPU 102 advances the processing to step S1407.

In step S1406, the CPU 102 causes the facial authentication processingunit 205 to reference the face dictionary DB 206 and thereby determineswhether or not the photographer has been identified from the face arearecognized in step S1405. If the photographer has been identified instep S1406, the CPU 102 advances the processing to step S1403, and ifthe photographer cannot be identified due to the face recognized in stepS1405 not being registered in the face dictionary DB 206, the CPU 102advances the processing to step S1407.

The processing in step S1407 is similar to that in step S503 in FIG. 5.

Although the present embodiment has described an example in whichauthentication of the photographer is performed based on peoplerecognized in the main camera image in the case where the photographercannot be identified in the sub-camera image, a person who has beenimmediately previously authenticated as the photographer can also besuccessively deemed to be the photographer, as will be described below.

FIG. 15 is a flowchart showing processing by which a person who has beenimmediately previously authenticated as the photographer is deemed to bethe photographer in the case where the photographer cannot be identifiedat the time of shooting, in the image capturing apparatus in the fourthembodiment.

In FIG. 15, in step S1501, the CPU 102 resets the result of thepreviously performed facial authentication processing with respect tothe photographer. Specifically, the name and the face dictionary data ofthe photographer that have been loaded into the memory 108 areinitialized. Initialization processing is performed at the time whenthere is thought to be a change in photographer, such as when a powerswitch is turned on or when initial settings are configured, and can beomitted in the case where the same photographer successively shoots aplurality of images.

In step S1502, the CPU 102 performs facial authentication processingwith respect to the photographer using any of the above-describedmethods, advances the processing to step S1503 if the photographer isauthenticated, and advances the processing to step S1504 if not.

In step S1503, the face dictionary for the photographer identified instep S1502 is loaded from the face dictionary DB 206 into the memory108, and the CPU 102 can reference the face dictionary using the facialauthentication processing unit 205.

On the other hand, in step S1508, since the photographer was notauthenticated in step S1502, the CPU 102 loads the face dictionary forthe photographer who was authenticated in the previous processing intothe memory 108. Thus, the face dictionary for the photographer at thetime when the facial authentication was previously performed is retainedunless authentication of a new photographer is performed.

The processing in step S1504 and subsequent steps is the same as that instep S504 and subsequent steps in FIG. 5.

As described above, if the photographer is not identified in step S1502,the CPU 102 advances the processing to step S1504 without changing thecurrent state and prepares for shooting. That is to say, the state thatwas set for the immediately previously authenticated photographer willbe maintained unless a new photographer is, identified. Thus, theimmediately previously authenticated photographer can be used as thephotographer at the time of shooting.

Fifth Embodiment

Next, processing for registering a face dictionary performed by an imagecapturing apparatus in a fifth embodiment will be described.

In the present embodiment, processing for registering different nameswith respect to the same object in face dictionaries for respectivephotographers is performed.

As described with reference to FIG. 4, in the face dictionary in thepresent embodiment, the name of an object having a common feature amount403 that is managed with the same ID 402 can be registered in the tables405 to 407 prepared for the respective photographers. That is to say,names that differ depending on the photographer can be registered withrespect to the same object in the face dictionary DB.

Therefore, a method by which a photographer registers a new object namein the face dictionary for that photographer will be described below.

FIG. 16 shows an exemplary operation screen 1601 displayed on a liquidcrystal panel 902 in FIG. 9, and the operation screen 1601 is displayedby a user pressing down the menu button 907 or the four-direction button903 in FIG. 9 when the above-described facial authentication processingis executed with respect to an object.

On the operation screen 1601, reference numeral 1602 denotes an objectname input box, and if an object has already been registered, the namethereof is displayed in this box, and if not, the box is displayed in anempty state. The user can input a new or changed object name in the nameinput box 1602 such as a touch panel. Reference numeral 1603 denotes areturn button, and upon this button being pressed, the input content inthe name input box 1602 is discarded and the registration processing iscancelled. Reference numeral 1604 denotes an OK button, and upon thisbutton being pressed, the input content in the name input box 1602 isreflected in the face dictionary for the photographer who is currentlyoperating, and the registration processing can be ended.

FIG. 17 is a flowchart showing processing in which a photographerregisters the name of an object in the face dictionary for thisphotographer, performed by the image capturing apparatus in the fifthembodiment.

In FIG. 17, in step S1701, upon the CPU 102 confirming that the user hasregistered the name of an object in the name input box 1602 on theoperation screen 1601 in FIG. 16 and the OK button 1604 has beenpressed, the CPU 102 advances the processing to step S1702 and checksthe content of the face dictionary for the photographer.

In step S1703, the CPU 102 determines whether or not the name that wasinput in the name input box 1602 has already been registered in the facedictionary for the photographer, advances the processing to step S1704if the name has been registered, and advances the processing to stepS1707 if not.

In step S1704, the CPU 102 displays information such as characters forprompting the user to change the registered name on the operation screen1601. In this case, the name that has already been registered isdisplayed in the name input box 1602 on the operation screen 1601, andcan be changed by means of editing by the user.

In step S1705, the CPU 102 determines whether or not the content of thename input box 1602 has been changed, advances the processing to stepS1706 if the content has been changed, and ends the processing if theexisting content is to retain as-is due to the return button 1603 beingpressed or for other reasons.

On the other hand, if the name has not been registered in the facedictionary for the photographer in step S1703, the CPU 102 advances theprocessing to step S1707 and checks the content of the face dictionariesfor other photographers.

In step S1708, if the name has not been registered in the facedictionaries for other photographers either, the CPU 102 advances theprocessing to step S1709, and advances the processing to step S1710 ifit has been registered.

In step S1709, the CPU 102 generates a new ID 402 for the input name,and in step S1710, the CPU 102 adds the new ID in the face dictionary DB401, registers the input name in the name input box 1602 on theoperation screen 1601, and ends the processing.

If the name has been registered in a face dictionary for anotherphotographer in step S1708, the CPU 102 omits generation of the ID 402in step S1709, newly registers the name that was input in the name inputbox 1602 in the face dictionary for the photographer in step S1710, andends the processing.

As described above, according to the present embodiment, different namescan be individually registered with respect to the same object in theface dictionaries for the respective photographers.

Sixth Embodiment

Next, processing for copying a face dictionary performed by an imagecapturing apparatus in a sixth embodiment will be described.

In the present embodiment, processing for copying a face dictionary fora photographer to another device such that the face dictionary isavailable in the other device is performed.

FIGS. 18A and 18B show exemplary operation screens displayed in adisplay unit of a copy source device, and FIGS. 19A and 19B showexemplary operation screens displayed in a display unit of a copydestination device. Although the present embodiment will describe amethod for copying a face dictionary in the copy source device using amemory card such as an SD (registered trademark) card as a recordingmedium from/to which data can be read/written also by the copydestination device, the face dictionary data may alternatively betransferred from the copy source to the copy destination using othercommunication module including wired and wireless communicationfunctions.

An operation screen 1801 in FIG. 18A is displayed on a liquid crystalpanel 902 in FIG. 9 upon a user designating a copy of a face dictionarywith a menu button 907 or a four-direction button 903 in FIG. 9.Reference numeral 1802 denotes a list of the face dictionaries for therespective photographers managed in the copy source device, and the usercan select the face dictionary for a desired photographer. Referencenumeral 1803 denotes a return button for interrupting the currentoperation, and reference numeral 1804 denotes an OK button forreflecting the operations performed thus far.

As shown in FIG. 18A, upon the user selecting the photographer X, whichis denoted by reference numeral 1802, on the operation screen 1801 andtouching the OK button 1804, the face dictionary data for the selectedphotographer X is copied to a memory card inserted in a card slot, via amemory interface 109. After copying is finished, the screen is switchedto an operation screen 1805 shown in FIG. 18B, and information such as amessage for prompting the user to insert the memory card in the copydestination device is displayed. The operation on the copy source deviceis finished by the user touching the OK button 1806.

An operation screen 1901 in FIG. 19A is displayed when the user insertsthe memory card in the copy destination device. Information such as amessage for prompting the user to select whether or not to copy the facedictionary data stored in the memory card is displayed on the operationscreen 1901. Copying of the face dictionary data to the copy destinationdevice is completed by the user touching an OK button 1903, whereascopying is not performed if the user touches a return button 1902. Whencopying is complete, the screen is switched to an operation screen 1904shown in FIG. 19B, information such as a message for notifying the userthat copying of the face dictionary data is complete is displayed, andthe operation on the copy destination device is finished by the usertouching the OK button 1903.

FIG. 20 shows an exemplary configuration of the face dictionary datathat is copied from the copy source device to the memory card in thesixth embodiment.

In FIG. 20, reference numeral 2001 denotes the face dictionary data forthe photographer X selected on the operation screen 1801. Referencenumeral 2002 denotes ID numbers of objects, reference numeral 2003denotes feature amounts of the faces of the objects, and referencenumeral 2004 denotes units of people (objects) for the selectedphotographer based on the association between the feature amounts of thefaces of the objects and the names registered in the face dictionary.

The face dictionary data 2001 for the selected photographer X isconstituted by extracting pieces of the information for the photographerX from the face dictionary DB 401 shown in FIG. 4, and the facedictionary data of unselected photographers is excluded.

FIG. 21 is a flowchart showing processing for copying the facedictionary performed by the image capturing apparatus in the sixthembodiment.

In FIG. 21, upon the user selecting a photographer in the copy source onthe operation screen 1801 in FIG. 18A in step S2101, the CPU 102 copiesthe name of the selected photographer and the face dictionary data 2001into the memory card in step S2102.

In step S2103, the memory card to which the face dictionary data hasbeen copied is inserted in the copy destination device, and upon aninstruction to write the face dictionary data onto the copy destinationdevice being given on the operation screen 1901, it is determined instep S2104 whether or not the face dictionary data of the samephotographer has been registered in the face dictionary DB 401 in thecopy destination device. If the face dictionary data for the samephotographer has been registered, the processing proceeds to step S2106,and if not, the processing proceeds to step S2105.

In step S2105, in the copy destination device, the face dictionary ofthe unregistered photographer is newly generated in the face dictionaryDB 401.

In step S2106, in the copy destination device, it is determined whetheror not the feature amount of the face of an object that is notregistered in the face dictionary DB in the copy destination deviceexists in the feature amounts 2003 of the faces of objects in the facedictionary data 2001 stored in the memory card. If no unregisteredobject exists, the processing proceeds to step S2108, and if anunregistered object exists, the processing proceeds to step S2107.

In step S2107, in the copy destination device, a new ID number is givento the unregistered object, and the feature amount of the face of thisobject is added to the face dictionary DB.

In step S2108, in the copy destination device, the face dictionary datastored in the memory card is copied.

As described above, according to the present embodiment, it is possibleto copy only the face dictionary for the photographer selected by theuser to another device so as to be available therein.

OTHER EMBODIMENTS

Embodiments of the present invention can also be realized by a computerof a system or apparatus that reads out and executes computer executableinstructions recorded on a storage medium (e.g., non-transitorycomputer-readable storage medium) to perform the functions of one ormore of the above-described embodiments of the present invention, and bya method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiments. The computer may comprise one or more of acentral processing unit (CPU), micro processing unit (MPU), or othercircuitry, and may include a network of separate computers or separatecomputer processors. The computer executable instructions may beprovided to the computer, for example, from a network or the storagemedium. The storage medium may include, for example, one or more of ahard disk, a random-access memory (RAM), a read only memory (ROM), astorage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blue-ray Disc(BD)™), a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2013-100705, filed May 10, 2013 which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image capturing apparatus, comprising: a first image capturing unit configured to capture an image of a photographer; a second image capturing unit configured to capture an image of a person other than the photographer; a storage unit configured to store a first dictionary corresponding to a first photographer and a second dictionary corresponding to a second photographer, wherein in the first dictionary and the second dictionary face feature information for authenticating a plurality of persons included in a captured image by the second image capturing unit and name information corresponding to an authenticated person are registered, and different name can be registered in each dictionary even if face feature information of the same person are registered in the first dictionary and the second dictionary; a recording unit configured to store image data generated by the second image capturing unit; a processor; and a memory configured to store at least one program executable by the processor, at least one program having instructions that, when the program is executed by the processor, cause the processor to: identify the photographer from the image captured by the first image capturing unit using facial recognition; select a dictionary corresponding to the identified photographer; authenticate one or more persons in the image captured by the second image capturing unit using the selected dictionary; display a frame so as to surround a face of a person in a through image captured by the second image capturing unit by using the face feature information of the plurality of persons stored in the selected dictionary, and record, as attribute information for the image data, face feature information and name information corresponding to the authenticated person that is included in the selected dictionary, in the recording unit.
 2. The apparatus according to claim 1, wherein the program includes instructions that cause the processor to display the image data generated by the second image capturing unit, wherein the name information corresponding to the authenticated person that is included in the selected dictionary is displayed in the vicinity of an image.
 3. The apparatus according to claim 2, wherein the program includes instructions that cause the processor to display a frame so as to surround a face of a person in the captured image by the second image capturing unit and to display a name of the person in the vicinity of the frame, based on the result of the authentication.
 4. The apparatus according to claim 1, wherein the program includes instructions that cause the processor to read biological information of the photographer, and specify the photographer using the read biological information.
 5. The apparatus according to claim 1, wherein if the photographer is not identified, the authentication is performed using the dictionary of a photographer who was identified in previous processing.
 6. The apparatus according to claim 1, wherein if the photographer is not identified, the authentication is performed using the face feature information registered in the second dictionary.
 7. The apparatus according to claim 1, wherein the selection is performed by designating the name of the photographer.
 8. The apparatus according to claim 1, wherein the first or second dictionary can be copied to another device.
 9. A control method of an image capturing apparatus, comprising: capturing an image of a photographer using a first image capturing unit; capturing an image of a person other than the photographer using a second image capturing unit; storing a first dictionary corresponding to a first photographer and a second dictionary corresponding to a second photographer, wherein in the first dictionary and the second dictionary face feature information for authenticating a plurality of persons included in a captured image by the second image capturing unit and name information corresponding to an authenticated person are registered, and different name can be registered in each dictionary even if face feature information of the same person are registered in the first dictionary and the second dictionary; storing image data generated by the second image capturing unit; identifying the photographer from the image captured by the first image capturing unit using face recognition; selecting a dictionary corresponding to the identified photographer; authenticating one or more persons in the image captured by the second image capturing unit using the selected dictionary; displaying a frame so as to surround a face of a person in a through image captured by the second image capturing unit by using the face feature information of the plurality of persons stored in the selected first or second dictionary, and recording, as attribute information for the image data, face feature information and name information corresponding to the authenticated person that is included in the selected dictionary.
 10. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a control method comprising: capturing an image of a photographer using a first image capturing unit; capturing an image of a person other than the photographer using a second image capturing unit; storing a first dictionary corresponding to a first photographer and a second dictionary corresponding to a second photographer, wherein in the first dictionary and the second dictionary face feature information for authenticating a plurality of persons included in a captured image by the second image capturing unit and name information corresponding to an authenticated person are registered, and different name can be registered in each dictionary even if face feature information of the same person are registered in the first dictionary and the second dictionary; storing image data generated by the second image capturing unit; identifying the photographer from the image captured by the first image capturing unit using face recognition; selecting a dictionary corresponding to the identified photographer; authenticating one or more persons in the image capture by the second image capturing unit using the selected dictionary; displaying a frame so as to surround a face of a person in a through image captured by the second image capturing unit by using the face feature information of the plurality of persons stored in the selected first or second dictionary, and recording, as attribute information for the image data, face feature information and name information corresponding to the authenticated person that is included in the selected dictionary. 